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Description 



SYSTEM AND METHOD FOR 
ANNOTATING DOCUMENTS 

Background of Invention 

[0001] The illustrative embodiments described in the present ap- 
plication are useful in systems including those for docu- 
ment modification and workflow and more particularly are 
useful in systems including those for collaborative anno- 
tation of documents. 

[0002] Documents are often created by groups of people that 
collaborate through iterations of additions, changes and 
suggestions. Team collaboration is often used when cre- 
ating or maintaining documents. A principle author may 
create an original document. Similarly, a principle author 
may create an initial draft of a document. The principle 
author typically uses a processing system such as a per- 
sonal computer with document creation software in- 
stalled. Once created, the document may be circulated to 
several team members for annotation. Typically, hardcopy 



paper printouts are circulated to the team members. The 
team members then each mark the document as needed 
and forward the annotations to the principle author or an- 
other designated person so that the updates and com- 
ments may be processed. 

[0003] The processing system typically includes a personal com- 
puter such as an INTEL PENTIUM ®based personal com- 
puter using the MICROSOFT WINDOWS XP ©operating sys- 
tem. A physical hardcopy printout of the electronic master 
version of the document may be created using a laser or 
inkjet printer. Computer systems are typically used to edit 
and publish each revision of the document. The docu- 
ments are often word processor data files used with a 
word processor and the computer systems typically in- 
clude word processing programs such as MICROSOFT 
®WORD®(hereinafter "Word") manufactured by Microsoft 
Corporation of One Microsoft Way, Redmond, Washington. 
However, the documents may also include, but are not 
limited to, business reports, contracts, legal briefs, arti- 
cles, books, speeches, reports, slide show presentations, 
financial reports, technical drawings, sales, marketing and 
advertising materials. 

[0004] Accordingly, other programs such as VISIO®and POWER- 



POINT®available from MICROSOFT ®may be used. Many 
additional programs and systems may be used including 
Computer Aided Design (CAD) systems. 

[0005] | n a traditional document collaboration effort, a principle 
author or editor creates an original document draft. The 
copies of the draft are circulated to the group and a mem- 
ber of the group is assigned the task of updating the draft 
to a new version. The group members typically provide 
change suggestions or comments by annotating the paper 
copy and returning it to the editor. The task of updating is 
usually performed by the principle author or editor and 
may include adding information, deleting information 
and/or changing information. There may be many hard- 
copy comment versions to reconcile and the process may 
be cumbersome. Furthermore, two or more of the sugges- 
tions may present a conflict and the member updating the 
draft may decide to enter the competing versions as alter- 
natives or resolve the conflict. The process may be itera- 
tive as the editor circulates subsequent drafts for com- 
ment until the document is finalized. 

[0006] The draft versions are typically hand-delivered or circu- 
lated by mail. More recently, document teams have begun 
using word-processing systems such as WORD that in- 



elude some collaborations features. For example, draft 
versions of WORD documents may be distributed elec- 
tronically. The team members may enter suggested 
changes and comments using the Track Changes feature 
of WORD. The editor then decides which changes to ac- 
cept. The editing/annotation process may be accom- 
plished in a round-robin fashion, with one single elec- 
tronic copy circulated to each team member in a serial 
fashion. Such a system requires that the team members 
utilize a computer equipped with WORD and may also re- 
quire that only one team member at a time manually 
check-out the document to enter suggestions. 
[0007] such prior art annotation methods are cumbersome, time- 
consuming and error-prone when the number of recipi- 
ents is more than a very small number. 
Summary of Invention 

[0008] The present application describes systems and methods 
for providing a document annotation manager system for 
use in team collaboration. Using digital paper, an elec- 
tronic document is associated with the special digital pa- 
per that is used for hardcopy printouts. Each team mem- 
ber may annotate the document using a digital pen. The 
multiple annotations are then each displayed with the 



electronic version of the original draft together with an 
identifier listing the particular team member and a time 
stamp for the annotation. 

[0009] a special printer is used to print both the digital paper 

and the document so that the electronic master document 
is associated with the printed hardcopy. The printer prints 
a digital paper identifier such as a carbon based dot pat- 
tern to provide a unique pattern space. The printer also 
prints the hardcopy document on the paper so that the 
electronic version of the document is associated with that 
particular pattern space. In an alternative, a traditional 
printer is used and an association is established between 
the digital paper and the electronic document. 

[0010] | n another alternative, editing symbols are utilized to pro- 
cess annotations in order to automatically incorporate 
changes to the electronic master document file. Addition- 
ally, a hierarchal permission system is utilized to deter- 
mine which team members may change the document and 
which members may only enter comments. Furthermore, if 
two team members with change authority attempt to 
change the same paragraph, the changes from the high- 
est-ranking team member are entered and the others are 
converted into comments. 



[0011] | n another alternative, a digital paper scanner is used 

when printing the hardcopies in order to associate a par- 
ticular digital paper identifier with the electronic master 
document. The digital paper identifier may be a unique 
pattern space that is scanned by the digital pen. In order 
to accommodate multiple page documents, each page of 
the digital paper may use a different pattern space and 
may be scanned as printed. Alternatively, the digital paper 
sheets may be provided in sequence and only the first 
page scanned. In that way, each printed page of the hard- 
copy document will be associated with a different digital 
paper pattern space. 

[0012] | n y e t another alternative embodiment, the document an- 
notation manager is utilized to audit the performance of a 
document reviewer such as a person that is grading pa- 
pers. The system may also be used to audit time records 
of a professional editor that is reviewing a work. The re- 
viewer is instructed to check passages or paragraphs dur- 
ing the review and the time-stamp subsystem of the an- 
notation manager records the time stamp entries for each 
annotation as the reviewer proceeds through the docu- 
ment. Accordingly, an audit trail is created that may be 
manually or automatically audited for compliance with a 



set of review rules. Additionally, the audit system may be 
utilized to provide non-repudiated evidence of document 
review such as by establishing review of financial reports 
by an audit manager to the satisfaction of the Chief Fi- 
nancial Officer of a large publicly traded corporation. 
Brief Description of Drawings 

[0013] FIG. 1 is a schematic representation of an annotation 

management system for creating digital paper hardcopies 
associated with electronic documents according to an il- 
lustrative embodiment of the present application. 

[0014] FIG. 1A is a flow chart showing a hardcopy creation pro- 
cess according to another illustrative embodiment of the 
present application. 

[0015] FIG. 2 is a schematic representation of a special printer 
used with the annotation management system of FIG. 1. 

[0016] FIG. 3 is a schematic representation of an annotation 

management system for receiving annotations from digital 
pens according to an illustrative embodiment of the 
present application. 

[0017] FIG. 3A is a flow chart showing an annotation manage- 
ment process according to another illustrative embodi- 
ment of the present application. 

[0018] FIG. 4 is a schematic representation of an illustrative doc- 



ument used in accordance with an embodiment of the 
present application. 

[0019] FIG. 5 is a schematic representation of an illustrative doc- 
ument used in accordance with an embodiment of the 
present application. 

[0020] FIG. 6 is a flow chart showing a hierarchal permission pro- 
cess according to another illustrative embodiment of the 
present application. 

[0021] FIG. 7 is a flow chart showing a process for reviewer audit 
according to another illustrative embodiment of the 
present application. 

[0022] FIG. 8 is a flow chart showing a process for legal review 
according to another illustrative embodiment of the 
present application. 
Detailed Description 

[0023] illustrative embodiments describing systems and methods 
for the collaborative annotation of documents are de- 
scribed. The illustrative embodiments are described with 
reference to the collaborative annotation of a single page 
legal document, but may be applied to other applications 
including multi-page documents. 

[0024] Digital pens allow a user to capture or digitize handwrit- 
ing or pen strokes that the user writes on a medium such 



as a piece of paper by using a processor such as a per- 
sonal computer. Digital pen systems include the Sony- 
Ericsson CHA-30 CHATPEN and ANOTO paper available 
from Anoto AB of Sweden. The CHATPEN utilizes a BLUE- 
TOOTH transceiver in order to communicate with a local 
processor. The ANOTO paper includes a grid for encoding 
position information that is detected by the CHATPEN. 
Additional information may be captured including infor- 
mation related to pressure, speed and pen attitude. The 
additional information includes biometric information that 
may be used to identify or authenticate a particular user. 
[0025] The ANOTO digital paper is encoded with a dot pattern 
that allows an ANOTO system to track a CHATPEN on the 
digital paper. The CHATPEN includes a camera that recog- 
nizes the dot pattern and resolves it to a pattern location. 
The CHATPEN sends pen stroke data, position data and 
biometric data to an application. Typically, the CHATPEN 
uses a BLUETOOTH transceiver and a local co-processor 
attached to a network such as the Internet to send posi- 
tion data to an ANOTO Pattern Lookup Service (PLS). The 
PLS associates the particular pattern location with an ap- 
plication URL and sends the data to the user application. 
The ANOTO user application described herein is an anno- 



tation manager service. Typical ANOTO applications asso- 
ciate a pattern space with a generic function such as a 
memo pad. A Notepad application on the local notebook 
computer of the user then receives digital pen input asso- 
ciated with that generic application. However, such sys- 
tems do not associate a particular instance of an elec- 
tronic document with a portion of the pattern space. 

[0026] a System for Paper Identification Information to Associate 
a Printed Application With an Electronic Application is de- 
scribed in U.S. Published Patent Application No. US 
2003/0001020A1 by James P. Kardach. The system de- 
scribed therein uses identification information printed in a 
corner of the hardcopy representation at a location that 
does not include any portion of the printed pattern. The 
pen is said to process the identification information to as- 
sociate the generic pattern with a particular hardcopy rep- 
resentation. 

[0027] Annotation Manager System 

[0028] | n an illustrative embodiment of the present application, a 
system for managing document annotations is described. 
The system utilizes an ANOTO CHATPEN digital pen sys- 
tem and paper including an ANOTO pattern. However, 
other digital pen systems may also be utilized. 



[0029] The illustrative systems and methods described provide a 
document annotation manager system for use in team 
collaboration. Using digital paper, an electronic document 
is associated with the special digital paper and hardcopy 
printouts are created using the digital paper. Each team 
member may annotate the document using a digital pen. 
The multiple annotations are then each displayed with the 
electronic version of the original draft together with an 
identifier for the particular team member and a time 
stamp for the annotation. 

[0030] | n an illustrative embodiment, a special printer is used to 
print both the digital paper and the document so that the 
electronic master document is associated with the printed 
hardcopy. The printer prints a digital paper identifier such 
as a carbon based dot pattern to provide a unique pattern 
space. The printer also prints the hardcopy document on 
the paper so that the electronic version of the document is 
associated with that particular pattern space. 

[0031] Referring to FIG. 1, an illustrative annotation management 
system for creating digital paper hardcopies associated 
with electronic documents according to the present appli- 
cation is shown. The application subsystem prints docu- 
ment copies for use with an annotation manager system. 



[0032] The annotation manager system allows users to maintain 
a single centralized annotated version among the team 
members and ensures that annotations are reflected on 
the master document in a timely manner. The annotations 
may be updated in real time. As annotations are made on 
the printed hardcopies, the original electronic instance is 
kept up to date automatically. Metadata relating to the 
annotations is captured and presented in the electronic 
version of the document. For example, the time and date 
of each annotation along with an identifier of the author 
of the comments are provided. 

[0033] | n an illustrative embodiment of the present application, a 
special printer 110 or raster plotter is used. The system 
uses ANOTO Digital Pen Functionality to print on licensed 
ANOTO digital paper. A master electronic document exists 
as a WORD file in a central repository 162 of an annota- 
tion manger system 160. Alternatively, the WORD file may 
reside in another location such as a document manage- 
ment system. 

[0034] Documents such as WORD data files are created using a 
word processing system. An instance of the WORD docu- 
ment is stored in the central repository 162 as an elec- 
tronic master file. The repository 162 is connected to the 



annotation manager server 164 using an Ethernet network 
connection 166. Alternative topologies may be used and 
the repository 162 may be stored locally with the server 
164. Multiple digital hardcopies 125 of the document are 
printed using digital paper 122. Printer 110 may concur- 
rently encode paper 120 with digital pattern information 
when printing the hardcopy documents 125. Alternatively, 
paper 120 may be preprinted with digital pattern informa- 
tion and the paper is associated with the document. 

[0035] | n this system, a special printer 110 prints multiple hard- 
copies 125 of the original version of the document. The 
printer 110 is connected to a network 130 such as the In- 
ternet using communications channel 132. The hard- 
copies 125 are then periodically distributed to the collab- 
oration team members. Alternatively, each member has a 
special printer for printing hardcopies 125 and receives 
only an electronic copy of the master document. The indi- 
vidual copy is then printed locally. 

[0036] Each team member has a digital pen such as the ANOTO 
CHATPEN that they use to provide feedback and com- 
ments. When a team member annotates the hardcopy 125, 
the pen strokes are written on the hardcopy 125. The pen 
strokes are also stored in the CHATPEN along with meta- 



data that includes time and date information as well as 
biometric information. Hardcopies 125 include a check- 
box to indicate that the user is done annotating the docu- 
ment. The annotations are then sent to the annotation 
manager system 160. 
[0037] The ANOTO pattern lookup service (PLS) 140 is connected 
to network 130 using communications 134. The PLS 140 
provides the URL of the Annotation Manager System 160. 
The annotation manager system 160 is connected to the 
network 130 using communications channel 136 and the 
annotation manager system program executes on server 
164. 

[0038] There are several team member/user identification 

schemes that may be used. In one scheme, an identical 
pattern is used for each page of the several copies of the 
document, but separate uniquely identifiable pens are 
used to identify a particular user as the source of certain 
annotations. In another scheme, each copy of the docu- 
ment is printed using different unique patterns and 
generic digital pens are used. Since each user received a 
uniquely identifiable copy, the source of annotations may 
be determined. In yet another scheme, individual patterns 
and uniquely identifiable individual pens may be used to 



provide a more secure system. For example, the user 
would be able to use her individual pen with only her indi- 
vidual copy of the document. 

[0039] | n an alternative, the annotations are sent to the annota- 
tion manager system 160 when they are made. In another 
alternative, a timeout counter is used to determine when a 
user is finished annotating a document. In another alter- 
native, the ANOTO pattern lookup service is provided lo- 
cally with the annotation manager server 164. Further- 
more, in yet another alternative, the pattern may be 
uniquely associated with a specific user who receives a 
specific unique and identifiable copy of the document. 

[0040] Referring to FIG. 1A, a process for creating hardcopies as- 
sociated with digital paper according to an embodiment of 
the present application is shown. 

[0041] | n s t e p igrj, the user picks a document to print for collab- 
oration. In step 192, the document is associated with a 
particular pattern on a digital paper. In step 194, the 
hardcopies are printed and, in step 196, the hardcopies 
are distributed to team members. 

[0042] in an alternative, each copy is printed using a unique pat- 
tern so that each user is provided a unique copy of the 
document that is associated with that particular user. In 



such instances, the underlying document is associated 
with several patterns in a one to many association. 

[0043] Referring to FIG. 2, a special printer used with the annota- 
tion management system as a hardcopy printing subsys- 
tem 210 is shown. A local processor 250 is connected to 
the annotation manager system and is connected to the 
printer 200 using communications channel 252. The 
printer 200 includes a pen cartridge 220 including a car- 
bon based ink pen 222 that is used to print an ANOTO 
pattern. The cartridge 220 also includes ANOTO Pen 224 
that is used with traditional ink to print the document 
contents on the hardcopy 225. 

[0044] | n t his embodiment, the ANOTO pen 224 is used to asso- 
ciate the physical location based version of the document 
with the ANOTO pattern. For example, the printer 200 
first prints a unique ANOTO pattern on a sheet of paper 
using a carbon-based ink pen 222 from the carousel. 
Next, the ANOTO pen 224 is selected from the carousel 
and used to write the document on the pattern. The AN- 
OTO pen 224 sends pen stroke information to a back-end 
application that then associates the document with the 
actual printing location on the pattern. 

[0045] | n an alternative, the local processor 250 uses a WYSIWYG 



algorithm with a print driver to determine where individual 
document elements will be printed on the paper. In such a 
system, the pen used to print the document is a standard 
ink pen. The local processor also maintains the geo- 
graphic location of the pattern being printed on the paper. 
The system then associates the two layouts to allow the 
system to associate a particular document element with a 
particular portion of the pattern space. 
[0046] | n y e t another alternative, the local processor receives a 
document for use with the centralized annotation man- 
ager system. The local processor associates a pattern with 
the document and prints the local copy of the document 
for the particular user by using locally generated ANOTO 
pattern to create a localized copy of the document. In 
such a system, the local processor selects a pattern for 
the local user and the particular copy is associated with 
that user. Each user has a similar individualized local sys- 
tem. Each local system then manages recognizing annota- 
tion input from a user and sends that annotation data to 
the centralized system. In another alternative, the local 
system sends pattern information to the centralized sys- 
tem and the user input is processed centrally as discussed 
above. 



[0047] Referring to FIG. 3, an annotation management system 

300 for receiving annotations from digital pens according 
to an illustrative embodiment of the present application is 
shown. 

[0048] An ANOTO CHATPEN 310 uses a local laptop 320 as a col- 
located local processor connected using a BLUETOOTH 
personal area network 315. The laptop 320 is connected 
to a network 330 such as the Internet using communica- 
tions channel 332. The communications channel 332 is a 
wireless network but could alternatively be any communi- 
cations channel such as a wired Ethernet connection. An 
ANOTO Pattern Lookup Service PLS 340 is connected to 
network 330 using communications channel 334. The an- 
notation manager service 360 includes a central reposi- 
tory 362 and server 364 connected to network 330 using 
communications channel 336. Alternatively, each of the 
connections may utilize a single local Ethernet Local Area 
Network (LAN). 

[0049] pen 310 includes a processor, memory, ink, a camera or 
image sensor, a battery and a BLUETOOTH transceiver. It 
also includes biometric sensors. Laptop 320 includes a 
mobile Pentium 4 processor. The server processors 340, 
364 are geographically and load balanced application 



servers including systems available from Sun Microsys- 
tems. The storage servers use multiple location redundant 
backup systems. Alternatively, other appropriate wireless 
and wired networks and connections may be utilized. Each 
user registers the ANOTO CHATPEN and provides a bio- 
metric sample such as a signature. In this embodiment, 
appropriate security is utilized including security at the 
application layer. 

[0050] The embodiments described herein may utilize biometric 
data for purposes such as identification and authentica- 
tion of a user. The pen 310 provides biometric data relat- 
ing to the pen strokes used including hand speed, pen tip 
pressure and the inclination angle between pen and pa- 
per. The pen 310 is assigned a unique identification code 
that is a unique serial number for the pen 310. Such code 
may also be used to identify a team member. 

[0051] Referring to FIG. 3A, a process for managing annotations 
associated with digital paper according to an embodiment 
of the present application is shown. 

[0052] | n step 390, team members write annotations on hard- 
copies. In step 392, the annotations associated with a 
particular electronic document are received by the anno- 
tation manager system with metadata. Each user has a 



unique pen and the annotations are associated with a par- 
ticular user. In step 394, the original electronic document 
is updated with the received annotations. 

[0053] | n an alternative, each document is printed with a unique 
pattern and a generic pen is used. Accordingly, the anno- 
tations may be associated with a particular user because 
the system stores information regarding which user re- 
ceived a particular copy of the document. 

[0054] Referring to FIG. 4, an illustrative document used in ac- 
cordance with an embodiment of the present application 
is shown. Document 425 is an original draft of a contract 
document. 

[0055] Referring to FIG. 5, an illustrative document used in ac- 
cordance with an embodiment of the present application 
is shown. Document 525 is a subsequent draft of the 
original draft of a contract document 425. The document 
includes annotations 530, 535 and 540. The document 
525 includes edit 545 that changes the state from New 
York to Connecticut. Annotations 530, 535, and 540 are 
anchored to their respective paragraphs. Accordingly, if a 
paragraph is moved, the annotation will move with it. An- 
notation 520 is anchored to the two words shown. Anno- 
tations 535 and 540 are anchored to the paragraph head- 



ings. 

[0056] | n the illustrative embodiments described herein, a single 
page contract is utilized to illustrate the operation of the 
system. However, multi-page documents may also be uti- 
lized. In another alternative, a bitmap printer such as a 
laser printer or an inkjet printer is utilized to print both an 
underlying ANOTO pattern and a document. 

[0057] Multiple copies of the same contract may be printed using 
the same ANOTO pattern space. Individual comments are 
identified using the unique identifying characteristics of 
the ANOTO CHATPEN as it annotates a particular copy of 
the contract. In a related alternative, each copy of the 
contract may be printed using a different ANOTO pattern 
space. In such an alternative system, the master document 
would be associated with each of the ANOTO pattern 
space areas used to print the multiple copies. 

[0058] | n another alternative embodiment, the same ANOTO pat- 
tern space is used for each piece of paper utilized by the 
current user of the annotation manager system. A trace- 
able identifier code is printed on the document hardcopy. 
The user then traces the code with the digital pen to cre- 
ate the association between that digital paper annotation 
session and the electronic master document. 



[0059] | n another alternative embodiment, the system determines 
if there is overlap or conflict amongst various annotations. 
If an overlap or conflict is detected, tactile/haptic feed- 
back is provided to the user by vibrating the pen. The user 
will then have an opportunity to review the central docu- 
ment using a computer before annotating that section. 

[0060] Annotation Manager System with Metadata 

[0061] | n another illustrative embodiment of the present applica- 
tion, a system for managing document annotations is de- 
scribed that also provides metadata for each annotation. 
The annotation manager system associates each annota- 
tion with an area of the original document. The digital pen 
provides additional information regarding the annotations 
as metadata. As shown in FIG. 5, annotations 530, 535, 
and 540 each include metadata regarding the author and 
time and date of the annotation. The metadata for edit 
545 is omitted from the figure but would be displayed in a 
comment box. 

[0062] Annotation Manager System with Annotation Anchoring 

[0063] in another illustrative embodiment of the present applica- 
tion, the annotation manager system receives annotation 
input from a digital pen such as the ANOTO CHATPEN 



system. 

[0064] a typical WORD document is presented in WORD in a What 
You See Is What You Get (WYSIWYG) format. Such a system 
considers the currently selected printer settings along 
with the traditional document formatting settings to de- 
termine the physical layout document presentation. The 
document is live and may be edited, as particular symbols 
and formatting codes are associated with a display loca- 
tion. In contrast, a bitmapped-scanned document exists 
only as an image file unless the character data is recog- 
nized using an optical character recognition (OCR) pro- 
gram. 

[0065] As shown in FIG. 5, annotation 530 has been processed 
using OCR before it was entered in the document 525. 
The annotation manager system uses annotation position 
data to determine which paragraph of the original docu- 
ment 425 is associated with the annotation. It then an- 
chors the annotation to that paragraph. 

[0066] Annotation Manager System with Editing and Reviewer Hi- 
erarchy 

[0067] | n another illustrative embodiment of the present applica- 
tion, the annotation manager system processes editing 
annotations using a hierarchal system. The annotations 



are parsed to recognize editing symbols that are used to 
automatically incorporate changes to the electronic mas- 
ter document file. For example, an annotation consisting 
of a line through a word will delete the word in the elec- 
tronic master copy. A carrot symbol is used to insert text. 
The inserted text is recognized using handwriting OCR. In 
an alternative, the editing commands are processed using 
the track changes functionality of WORD. 

[0068] Referring to FIG. 6, a flowchart of a hierarchal permission 
system according to an embodiment of the present appli- 
cation is shown. A hierarchal permission system is utilized 
to determine which team members may change the docu- 
ment and which members may only enter comments. Ad- 
ditionally, if two team members with change authority at- 
tempts to change the same paragraph, the changes from 
the highest-ranking team member are entered and the 
others are converted into comments. 

[0069] The editing system is an application running on the anno- 
tation manager infrastructure described above. In step 
610, a group of annotations are received. In step 620, the 
annotations are parsed and any control annotations are 
recognized. In step, 630, the system determines if any 
users having only comment privileges entered any editing 



commands. If so, in step 635, those editing commands 
are ignored and the edits are converted to comment an- 
notations. 

[0070] | n s tep 640, the system determines if there are any 

groups of editing commands that operate on the same 
portion of the original electronic master document. If so, 
in step 650, the edits are prioritized and only the one edit 
with highest priority is selected. In step 660, any edits 
that are not processed due to the hierarchy test are con- 
verted to annotation comments. 

[0071] Digital Paper Scanner Used for Association 

[0072] | n another alternative, a digital paper scanner is used 

when printing the hardcopies in order to associate a par- 
ticular digital paper identifier with the electronic master 
document. The digital paper identifier may be a unique 
pattern space that is scanned by the digital pen. In order 
to accommodate multiple page documents, each page of 
the digital paper may use a different pattern space and 
may be scanned as printed. Alternatively, the digital paper 
sheets may be provided in sequence and only the first 
page scanned. In that way, each printed page of the hard- 
copy document will be associated with a different digital 
paper pattern space. 



[0073] Referring again to FIG. 1, a traditional laser printer is uti- 
lized as an alternative to special printer 110. In order to 
associate a particular sheet of digital paper with the elec- 
tronic master document, a digital pen or other pattern 
scanner is used to identify the paper before it is placed in 
the traditional printer. Once the digital paper is identified, 
the hardcopy of the document is printed and the associa- 
tion with the digital paper is established in the annotation 
manager service database. 

[0074] Reviewer Audit System 

[0075] | n another illustrative embodiment of the present applica- 
tion, the annotation manager system is adapted to audit 
certain characteristics of a reviewer who is engaged in 
document review. 

[0076] Several groups of people routinely review documents writ- 
ten by others. For example, professors often use graders 
to grade papers or exams submitted in courses having a 
large number of students. Similarly, editors review mate- 
rial for articles and books. Furthermore, certain reviewers 
may be compensated based upon the amount of time 
spent reviewing a document. For example, an attorney 
may be compensated hourly for reviewing a draft appel- 
late brief. 



[0077] it may De advantageous to provide a report detailing the 
amount of time spent reviewing such a document. Ac- 
cordingly, in an alternative embodiment using any of the 
annotation manager systems described, a reviewer audit 
system is described. The document annotation manager is 
modified to audit the performance of a document reviewer 
such as a person that is grading papers or a professional 
editor that is reviewing a work. The reviewer is instructed 
to check passages or paragraphs during the review and 
the time-stamp system of the annotation manager 
records the time stamp entries for each annotation as the 
reviewer proceeds through the document. Accordingly, an 
audit trail is created that may be manually or automati- 
cally audited for compliance with a set of review rules. 

[0078] Referring to FIG. 7, a flowchart of a reviewer audit system 
according to an embodiment of the present application is 
shown. The system is an application running on the anno- 
tation manager infrastructure described above. In step 
710, an examination paper is printed on and associated 
with digital paper, leaving space for answers. The report 
includes grade boxes next to each exam question. In step 
720, the grader grades the paper using a digital pen. The 
grader considers that answer to each question and places 



a grade in the grade box. At step 730, the grader docks 
the digital pen so that the pen initiates a communication 
session with the reviewer audit system. At step 740, the 
reviewer audit system generates a grading report that de- 
scribes the amount of time spent reviewing each question 
by using the time stamps associated with the review. If 
multiple graders are used, a statistical analysis comparing 
the graders is provided. 

[0079] Optionally, at step 750, the reviewer audit system pre- 
pares a billing report listing the expected number of 
hours for each activity of the reviewer during a period 
along with the actual time spent. Such a report could be 
used in auditing the monthly bill of the reviewer. 

[0080] Non-Repudiated Legal Review 

[0081] Due to the occurrence of several high profile corporate 
scandals, the Sarbanes-Oxley Act of 2002 (the Act) im- 
posed certain duties and obligations on board members 
and other individuals representing certain companies. 

[0082] | n accordance with section 302 of the Act, the CEO and 
CFO of each issuer must prepare a statement to accom- 
pany each audit report to certify the appropriateness of 
the financial statements and disclosures contained in the 
periodic report. They certify that those financial state- 



merits and disclosures fairly present, in all material re- 
spects, the operations and financial condition of the is- 
suer. 

[0083] it may be advantageous to provide a certain level of cer- 
tainty that a particular individual reviewed a document. 
For example, a CFO may rely upon an audit manager to 
review documents. The CFO may desire a record of review 
of a particular document or group of documents that the 
audit manager could not repudiate. 

[0084] Referring to FIG. 8, a flowchart of a legal review system 
according to an embodiment of the present application is 
shown. Accordingly, in an alternative embodiment using 
any of the annotation manager systems described, a legal 
review non-repudiation review system is described. The 
system is an application running on the annotation man- 
ager infrastructure described above. In step 810, a finan- 
cial report is printed on and associated with digital paper. 
The report optionally includes check boxes after each 
paragraph or section. Additionally, a signature box is in- 
cluded at the end of the document. In step 820, the audit 
manager reviews the document using a digital pen. The 
audit manager checks the review check boxes if they are 
available during the review or checks the digital paper 



near each section or paragraph. 

[0085] At step 830, the audit manager signs the reviewed docu- 
ment in the signature box. At step 840, the audit manager 
docks the digital pen so that the pen initiates a communi- 
cation session with the legal review system. In an alterna- 
tive, the digital pen communicates with the legal review 
system through a real-time communications channel such 
as a BLUETOOTH link to a cellular telephone that commu- 
nicates with the legal review system. At step 850, the legal 
review system generates a review report that describes the 
level of authentication of the digital pen user and a report 
of time stamps associated with the review. 

[0086] Optionally, the legal review system will apply a set of rules 
in order to flag suspicious activity. For example, the legal 
review system may expect that the audit manager should 
spend at least two minutes reviewing a particular section. 
If the report shows that the boxes were checked in two- 
second intervals, a red flag will be reported. The levels of 
authentication used include at least a first level that in- 
cludes a unique pen identifier that is associated with a 
user and a second level that includes biometric authenti- 
cation of the user signature or writing sample against a 
known reference. 



[0087] Multiple Source Annotation Manager 

[0088] Commonly owned, co-pending U.S. patent application se- 
rial numberlO/389,498, filed March 14, 2003 describes a 
Method For Managing The Annotation Of Documents and 
is incorporated herein by reference. The embodiments of 
the application may be combined with the embodiments 
described herein to extend the annotation manager sys- 
tem to include multiple annotation sources. 

[0089] Commonly owned, co-pending U.S. patent application se- 
rial number 10/065,284, filed September 30, 2002 de- 
scribes methods and systems for remote form completion 
and is incorporated herein by reference. Commonly 
owned, co-pending U.S. patent application serial number 
10/065,281, filed September 30, 2002 describes methods 
and systems for creating and sending facsimiles using a 
digital pen and is incorporated herein by reference. 

[0090] The illustrative embodiments of the present application 
are described with reference to a word processing system 
such as the MICROSOFT® WORD system. However, the 
embodiments of the present application may be used with 
other document creation and editing systems such as CAD 
systems and presentation systems including MICROSOFT ® 
POWERPOINT. The systems described herein may be im- 



plemented using computer software development systems 
such as MICROSOFT VISUAL STUDIO ® and using program- 
ming languages such as C and C++ with appropriate li- 
braries. Similarly, a commercial product modification tool 
such as VISUAL BASIC APPLICATIONS (VBA) may be used to 
modify a commercial application such as WORD. Alterna- 
tively, an open source system such as OPEN OFFICE may 
be modified to perform the functions described herein. 
[0091] The processing systems used are typically INTEL PENTIUM 
®based personal computer, but other computers such as 
MACINTOSH APPLE computers may be used. Similarly, the 
processing systems may include, but are not limited to, 
data processors, keyboards, dictation equipment, drawing 
tablets, pointing devices, mobile computers, tablet com- 
puters, Personal Digital Assistants (PDA), cell phones and 
digital pens. 

[0092] The illustrative embodiments described herein utilize the 
ANOTO CHATPEN system. However, other digital pens 
may be utilized. Certain digital pen systems utilize an 
imaging device to scan or record an image of the pen 
stroke. Other digital pens use mechanical sensors in order 
to record a pen stroke. Digital pen systems may utilize 
positioning systems such as light-based scanning systems 



including infrared (IR) sources and detectors in order to 
determine an absolute or relative position of the pen. 
Those pen systems and the digital pen systems include 
the LOGITECH 10 pen system from Logitech of Freemont, 
California, the N-SCRIBE system available from Digital Ink 
of Wellesley, MA and the E-PEN system available from E- 
Pen InMotion of Matam, Haifa Israel may be utilized. Simi- 
larly, digital pointing device systems including the V-PEN 
system available from OTM Technologies of Herzliya Israel 
may be used. 

[0093] The present application describes illustrative embodi- 
ments of a system and method for annotating documents. 
The embodiments are illustrative and not intended to 
present an exhaustive list of possible configurations. 
Where alternative elements are described, they are under- 
stood to fully describe alternative embodiments without 
repeating common elements whether or not expressly 
stated to so relate. Similarly, alternatives described for el- 
ements used in more than one embodiment are under- 
stood to describe alternative embodiments for each of the 
described embodiments having that element. 

[0094] The described embodiments are illustrative and the above 
description may indicate to those skilled in the art addi- 



tional ways in which the principles of this invention may 
be used without departing from the spirit of the invention. 
Accordingly, the scope of each of the claims is not to be 
limited by the particular embodiments described. 



